The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis, we designed an international survey that was issued to all participants of challenges conducted in conjunction with the IEEE ISBI 2021 and MICCAI 2021 conferences (80 competitions in total). The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. A median of 72% challenge participants took part in the survey. According to our results, knowledge exchange was the primary incentive (70%) for participation, while the reception of prize money played only a minor role (16%). While a median of 80 working hours was spent on method development, a large portion of participants stated that they did not have enough time for method development (32%). 25% perceived the infrastructure to be a bottleneck. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures. 43% of the respondents reported that the data samples (e.g., images) were too large to be processed at once. This was most commonly addressed by patch-based training (69%), downsampling (37%), and solving 3D analysis tasks as a series of 2D tasks. K-fold cross-validation on the training set was performed by only 37% of the participants and only 50% of the participants performed ensembling based on multiple identical models (61%) or heterogeneous models (39%). 48% of the respondents applied postprocessing steps.

Deep learning classifiers provide the most accurate means of automatically diagnosing diabetic retinopathy (DR) based on optical coherence tomography (OCT) and its angiography (OCTA). The power of these models is attributable in part to the inclusion of hidden layers that provide the complexity required to achieve a desired task. However, hidden layers also render algorithm outputs difficult to interpret. Here we introduce a novel biomarker activation map (BAM) framework based on generative adversarial learning that allows clinicians to verify and understand classifiers decision-making. A data set including 456 macular scans were graded as non-referable or referable DR based on current clinical standards. A DR classifier that was used to evaluate our BAM was first trained based on this data set. The BAM generation framework was designed by combing two U-shaped generators to provide meaningful interpretability to this classifier. The main generator was trained to take referable scans as input and produce an output that would be classified by the classifier as non-referable. The BAM is then constructed as the difference image between the output and input of the main generator. To ensure that the BAM only highlights classifier-utilized biomarkers an assistant generator was trained to do the opposite, producing scans that would be classified as referable by the classifier from non-referable scans. The generated BAMs highlighted known pathologic features including nonperfusion area and retinal fluid. A fully interpretable classifier based on these highlights could help clinicians better utilize and verify automated DR diagnosis.

We explore unifying a neural segmenter with two-pass cascaded encoder ASR into a single model. A key challenge is allowing the segmenter (which runs in real-time, synchronously with the decoder) to finalize the 2nd pass (which runs 900 ms behind real-time) without introducing user-perceived latency or deletion errors during inference. We propose a design where the neural segmenter is integrated with the causal 1st pass decoder to emit a end-of-segment (EOS) signal in real-time. The EOS signal is then used to finalize the non-causal 2nd pass. We experiment with different ways to finalize the 2nd pass, and find that a novel dummy frame injection strategy allows for simultaneous high quality 2nd pass results and low finalization latency. On a real-world long-form captioning task (YouTube), we achieve 2.4% relative WER and 140 ms EOS latency gains over a baseline VAD-based segmenter with the same cascaded encoder.

Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License.

自2016年成立以来，Alexa奖计划使数百名大学生能够通过Socialbot Grand Challenge探索和竞争以发展对话代理商。挑战的目的是建立能够与人类在流行主题上连贯而诱人的代理人20分钟，同时达到至少4.0/5.0的平均评分。但是，由于对话代理商试图帮助用户完成日益复杂的任务，因此需要新的对话AI技术和评估平台。成立于2021年的Alexa奖Taskbot Challenge建立在Socialbot Challenge的成功基础上，通过引入交互式协助人类进行现实世界烹饪和做自己动手做的任务的要求，同时同时使用语音和视觉方式。这项挑战要求TaskBots识别和理解用户的需求，识别和集成任务和域知识，并开发新的方式，不分散用户的注意力，而不必分散他们的任务，以及其他挑战。本文概述了Taskbot挑战赛，描述了使用Cobot Toolkit提供给团队提供的基础架构支持，并总结了参与团队以克服研究挑战所采取的方法。最后，它分析了比赛第一年的竞争任务机器人的性能。

从数据集中生成新样本可以减轻额外的昂贵操作，增加的入侵程序并减轻隐私问题。当关注隐私时，这些新颖的样本在统计上是稳健的。此方法可以实现更好的数据共享实践，而无需与识别问题或偏见有关的对抗性攻击的缺陷。

深度估计是在机器人手术和腹腔镜成像系统中进行图像引导干预的关键步骤。由于对于腹腔镜图像数据很难获得人均深度地面真相，因此很少将监督深度估计应用于手术应用。作为替代方案，已经引入了仅使用同步的立体图像对来训练深度估计器。但是，最近的工作集中在2D中的左右一致性上，而忽略了现实世界坐标中对象的宝贵固有3D信息，这意味着左右3D几何结构一致性尚未得到充分利用。为了克服这一限制，我们提出了M3Depth，这是一种自我监督的深度估计器，以利用3D几何结构信息隐藏在立体声对中，同时保持单眼推理。该方法还消除了在至少一个立体声图像中通过掩码看不见的边界区域的影响，以增强重叠区域中的左图和右图像之间的对应关系。密集实验表明，我们的方法在公共数据集和新获取的数据集上的以前的自我监督方法都大大优先，这表明在不同的样品和腹腔镜上都有良好的概括。

我们研究了一种基于播种在随机节点上的个性化Pagerank矢量矩阵的简单嵌入技术。我们表明，该矩阵（1）的元素对数产生的嵌入与光谱嵌入具有重要意义的一类图形的光谱嵌入有关，因此可以对数据进行有用的表示，（2）可以为（2）完成（2）与网络的大小相比，整个网络或较小的部分，可以实现精确的本地表示形式，并且（3）使用相对较少的Pagerank向量。最重要的是，这种嵌入策略的一般性质打开了许多新兴应用，这些应用程序可能无法确定为基于Pagerank的亲戚。例如，类似的技术可以在来自HyperGraphs的Pagerank矢量上使用，以获取“光谱样”的嵌入。

通过一系列联邦举措和命令，美国政府一直在努力确保美国在AI中的领导。这些广泛的战略文件影响了美国空军美国部（DAF）等组织。DAF-MIT AI加速器是DAF和MIT之间的一项计划，以弥合AI研究人员与DAF任务要求之间的差距。DAF-MIT AI加速器支持的几个项目正在开发公共挑战问题，这些问题解决了许多联邦AI研究的重点。这些挑战是通过公开可用的大型AI-Ready数据集，激励开源解决方案，并为可以激发进一步研究的双重使用技术创建需求信号，来针对优先事项。在本文中，我们描述了正在开发的这些公共挑战以及它们的应用如何促进科学进步。

我们展示了一种物理感知的变压器，用于从具有不同分辨率，颜色空间，焦距，焦距和暴露的相机的基于特征的数据融合。我们还展示了使用开源计算机图形软件为变压器合成训练数据生成的可扩展解决方案。我们演示了具有不同光谱响应，瞬时视野和框架速率的阵列上的图像合成。